WE CLAIM; 



1 . An apparatus for determining a sequence of sub-word 
units representative of at least two words output by a 
word recognition unit in response to a common input word 
to be recognised, the apparatus comprising: 

means for receiving a first sequence of sub-word 
units representative of a first one of said at least two 
words and for receiving a second sequence of sub-word 
units representative of a second one of said at least two 
words ; 

means for aligning and for comparing sub-word units 
of the first sequence with sub-word units of the second 
sequence to form a number of aligned pairs of sub-word 
units; and 

means for determining a sequence of sub-word units 
representative of the received words in dependence upon 
the aligned pairs of sub-word units determined by said 
aligning and comparing means . 

2. An apparatus according to claim 1, wherein said 
determining means is operable to determine said sequence 
of sub-word units by determining, for each aligned pair 
of sub-word units, a sub-word unit that it confusingly 



65 

similar to the first and second sub-word units of the 
aligned pair. 

3. An apparatus according to claim 2, wherein said 
determining means comprises: 

first comparing means for comparing, for each 
aligned pair, the first sequence sub-word unit in the 
aligned pair with each of a plurality of sub-word units 
taken from a set of predetermined sub-word units, to 
provide a corresponding plurality of comparison scores 
representative of the similarities between the first 
sequence sub-word unit and the respective sub-word units 
of the set; 

second comparing means for comparing, for each 
aligned pair, the second sequence sub-word unit in the 
aligned pair with each of said plurality of sub-word 
units from the set, to provide a further corresponding 
plurality of comparison scores representative of the 
similarities between said second sequence sub-word unit 
and the respective sub-word units of the set; 

means for combining the comparison scores obtained 
when comparing the first and second sequence sub-word 
units in the aligned pair with the same sub-word unit 
from the set, to generate a plurality of combined 
comparison scores ; 
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■third comparing means for comparing , for each 
aligned pair, the combined comparison scores generated by 
said combining means for the aligned pair; and 

means for determining , for each aligned pair of sub- 
5 word units, a sub-word unit representative of the sub- 

word units in the aligned pair in dependence upon a 
comparison result output by said third comparing means 
O for the aligned pair, 

r 5 

Q 10 4. An apparatus according to claim 3, wherein said 

is 5 :? 

CO first and second comparing means are operable to compare 

= the first sequence sub-word unit and the second sequence 

=p sub-word unit respectively with each of the sub-word 

U1 units in said set of sub-word units. 

5. An apparatus according to claim 3, wherein said 
first and second comparing means are operable to provide 
comparison scores which are indicative of a probability 
of confusing the corresponding sub-word unit taken from 
20 the set of predetermined sub-word units as the sub-word 

unit in the aligned pair. 
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6. An apparatus according to claim 5, wherein said 
combining means is operable to combine the comparison 
scores in order to multiply the probabilities of 



confusing the corresponding sub-word unit taken from the 
set as the sub-word units in the aligned pair. 

7. An apparatus according to claim 6, wherein each of 
said sub-word units in said set of predetermined sub-word 
units has a predetermined probability of occurring within 
a sequence of sub-word units and wherein said combining 
means is operable to weigh each of said combined 
comparison scores in dependence upon the respective 
probability of occurrence for the sub-word unit of the 
set used to generate the combined comparison score. 

8. An apparatus according to claim 7, wherein said 
combining means is operable to combine said comparison 
scores by calculating: 

P(^\p r )p(d]\p r )p[p r ) 

where d} t and d 2 j are an aligned pair of first and 
second sequence sub-word units respectively; P(di|p r ) is 
the comparison score output by said first comparing means 
and is representative of the probability of confusing set 
sub-word unit p r as first sequence sub-word unit d* ± ; 
P(d 2 j|p r ) is the comparison score output by said second 
comparing means and is representative of the probability 
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of confusing set sub-word unit p r as second sequence sub- 
word unit d 2 j; and P(p r ) is a weight which represents the 
probability of set sub-word unit p r occurring in a 
sequence of sub-word units. 

5 

9. An apparatus according to claim 8, wherein said 
third comparing means is operable to identify the set 

en 

\D sub-word unit p r which gives the maximum combined 

Jp comparison score and wherein said determining means is 

CO 10 operable to determine said sub-word unit representative 

lit 

£n of the sub-word units in the aligned pair as being the 

O sub-word unit which provides the maximum combined 

nj comparison score. 

o 

15 10. An apparatus according to claim 6, wherein said 

comparison scores represent log probabilities and wherein 
said combining means is operable to multiply said 
probabilities by adding the respective comparison scores. 



20 11. An apparatus according to claim 3, wherein each of 

the sub-word units in said first and second sequences of 
sub-word units belong to said set of predetermined sub- 
word units and wherein said first and second comparing 
means are operable to provide said comparison scores 
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using predetermined data which relate the sub-word units 
in said set to each other, 

12. An apparatus according to claim 11 , wherein said 
5 predetermined data comprises, for each sub-word unit in 

the set of sub-word units, a probability for confusing 
that sub-word unit with each of the other sub-word units 
yg in the set of sub-word units. 

ffj 10 13. An apparatus according to claim 1, wherein said 

ffl aligning and comparing means comprises dynamic 

g programming means for aligning said first and second 

fij sequences of sub-word units using a dynamic programming 

technique. 
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14. An apparatus according to claim 13, wherein said 
dynamic programming means is operable to determine an 
optimum alignment between said first and second sequences 
of sub-word units. 

15. An apparatus according to claim 1, wherein each of 
said sub-word units represents a phoneme. 
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16. An apparatus according to claim 1, wherein said 
receiving means is operable to receive a third sequence 
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of sub-word unit representative of a third one of the 
words output by said word recognition unit and wherein 
said aligning and comparing means is operable to 
simultaneously align and compare the sub-word units of 
5 the first , second and third sequences of sub-word units. 

17. An apparatus according to claim 1, wherein said 
receiving means is operable to receive a third sequence 
of sub-word units representative of a third one of said 
10 words output by said recognition unit and wherein said 

aligning and comparing means is operable to align and 
compare two sequences of sub-word units at a time. 



18. An apparatus according to claim 1, further 
15 comprising a word to sub-word unit dictionary which is 

operable to receive the words output by said word 
recognition unit and to generate therefrom said sequences 
of sub-word units . 

20 19. An apparatus according to claim 1, further 

comprising means for annotating a data file using the 
sub-word units output by said determining means . 
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20. An apparatus according to claim 19, wherein said 
annotating means is operable to annotate said data file 



using said sub-word units and said words output by said 
word recognition unit . 

21. An apparatus according to claim 20, wherein said 
sequence of sub-word units and said words are combined to 
form annotation data for the data file. 

22. An apparatus according to claim 19, wherein said 
data file is one of: an audio data file, a video data 
file, an image data file or a text data file. 

23. An apparatus according to claim 1, wherein said word 
recognition unit comprises a speech recognition system. 

24. An apparatus according to claim 1, wherein said word 
recognition unit comprises a handwriting recognition 
system. 

25. An apparatus for determining a sequence of sub-word 
units representative of at least two words, the apparatus 
comprising : 

means for receiving a first sequence of sub-word 
units representative of a first word and for receiving a 
second sequence of sub-word units representative of a 
second word; 



10 
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means for aligning sub-word units of the first 
sequence with sub-word units of the second sequence to 
form a number of aligned pairs of sub-word units; and 

means for determining a sequence of sub-word units 
representative of the first and second sequences of sub- 
word units by determining, for each aligned pair of sub- 
word units , a sub-word unit that is confusingly similar 
to the first and second sub-word units of the aligned 
pair. 



26. An apparatus for determining a sequence of sub-word 
units representative of at least two words output by a 
word recognition unit in response to a common input word 
to be recognised, the apparatus comprising: 

15 means for receiving the words output by the word 

recognition unit; 

means for generating a sequence of sub-word units 
representative of each of the received words; 

means for aligning and comparing the sub-word units 

20 from each generated sequence of sub-word units to 

identify a number aligned groups of sub-word units; and 
means for determining a sequence of sub-word units 
representative of the received words in dependence upon 
the aligned groups of sub-word units determined by said 

2 5 aligning and comparing means. 



27. An apparatus for determining a sequence of sub-word 
units representative of at least two words output by a 
word recognition unit in response to a common input word 
to be recognised, the apparatus comprising: 

means for receiving the words output by the word 
recognition unit; 

means for generating a sequence of sub-word units 
representative of each of the received words; 

means for aligning and comparing the sub-word units 
from each generated sequence of sub-word units to 
identify a number aligned groups of sub-word units; and 

means for determining a sequence of sub-word units 
representative of the first and second sequences of sub- 
word units by determining, for each of the sub-word units 
in an aligned group, a sub-word unit that it confusingly 
similar to the sub-word units of the group. 

28. An apparatus for determining a sequence of sub-word 
units representative of at least two words output by a 
word recognition unit in response to a common input word 
to be recognised, the apparatus comprising: 

means for receiving a first sequence of sub-word 
units representative of a first one of said at least two 
words and for receiving a second sequence of sub-word 
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units representative of a second one of said at least two 
words; 

means for aligning and for comparing sub-word units 
of the first sequence with sub-word units of the second 
5 sequence to form a number of aligned pairs of sub-word 

units; 

^ first comparing means for comparing, for each 

]y aligned pair, the first sequence sub-word unit in the 

HF aligned pair with each of a plurality of sub-word units 

10 taken from a set of predetermined sub-word units, to 

provide a corresponding plurality of comparison scores 
y representative of the similarities between the first 

fU sequence sub-word unit and the respective sub-word units 

O of the set; 

15 second comparing means for comparing, for each 

aligned pair, the second sequence sub-word unit in the 
aligned pair with each of said plurality of sub-word 
units from the set, to provide a further corresponding 
plurality of comparison scores representative of the 
20 similarities between said second sequence sub-word unit 

and the respective sub-word units of the set; 

means for combining the comparison scores obtained 
when comparing the first and second sequence sub-word 
units in the aligned pair with the same sub-word unit 
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from the set, to generate a plurality of combined 
comparison scores ; 

third comparing means for comparing , for each 
aligned pair, the combined comparison scores generated by 
5 said combining means for the aligned pair; and 

means for determining, for each aligned pair of sub- 
word units, a sub-word unit representative of the sub- 
word units in the aligned pair in dependence upon a 
comparison result output by said third comparing means 
10 for the aligned pair. 

29, A method of determining a sequence of sub-word units 
representative of at least two words output by a word 
recognition unit in response to a common input word to be 
15 recognised, the method comprising: 

receiving a first sequence of sub-word units 
representative of a first one of said at least two words; 

receiving a second sequence of sub-word units 
representative of a second one of said at least two 
20 words; 

aligning and comparing sub-word units of the first 
sequence with sub-word units of the second sequence to 
form a number of aligned pairs of sub-word units; and 

determining a sequence of sub-word units 
25 representative of the received sequences of sub-word 
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units in dependence upon the aligned pairs of sub-word 
units determined in said aligning and comparing step. 



30. A method according to claim 29, wherein said 
5 determining step determines said sequence of sub-word 

units by determining, for each aligned pair of sub-word 
units, a sub-word unit that is confusingly similar to the 
first and second sub-word units of the aligned pair. 



10 31. A method according to claim 29 , wherein said 

determining step comprises: 

a first comparing step of comparing, for each 
aligned pair, the first sequence sub-word unit in the 
aligned pair with each of a plurality of sub-word units 

15 taken from a set of predetermined sub-word units, to 

provide a corresponding plurality of comparison scores 
representative of the similarities between the first 
sequence sub-word unit and the respective sub-word units 
of the set; 

20 a second comparing step of comparing, for each 

aligned pair, the second sequence sub-word unit in the 
aligned pair with each of said plurality of sub-word 
units from the set, to provide a further corresponding 
plurality of comparison scores representative of the 



similarities between said second sequence sub-word unit 
and the respective sub-word units of the set; 

combining the comparison scores obtained when 
comparing the first and second sequence sub-word units in 
the aligned pair with the same sub-word unit from the 
set, to generate a plurality of combined comparison 
scores; 

a third comparing step of comparing, for each 
aligned pair, the combined comparison scores generated in 
said combining step for the aligned pair; and 

determining, for each aligned pair, a sub-word unit 
representative of the sub-word units in the aligned pair 
in dependence upon a comparison result output from said 
third comparing step for the aligned pair. 

32. A method according to claim 31, wherein said first 
and second comparing steps compare the first sequence 
sub-word unit and the second sequence sub-word unit 
respectively with each of the sub-word units in said set 
of sub-word units . 

33. A method according to claim 31, wherein said first 
and second comparing steps provide comparison scores 
which are indicative of a probability of confusing the 
corresponding sub-word unit taken from the set of 
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predetermined sub-word units as the sub-word unit in the 
aligned pair. 

34. A method according to claim 33, wherein said 
combining step combines the comparison scores in order to 
multiply the probabilities of confusing the corresponding 
sub-word unit taken from the set as the sub-word units in 
the aligned pair. 

35. A method according to claim 34, wherein each of said 
sub-word units in said set of predetermined sub-word 
units has a predetermined probability of occurring within 
a sequence of sub-word units and wherein said combining 
step weighs each of said combined comparison scores in 
dependence upon the respective probability of occurrence 
for the sub-word unit of the set used to generate the 
combined comparison score. 

36. A method according to claim 35, wherein said 
combining step combines said comparison scores by 
calculating : 




# 
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where d x ± and d 2 j are an aligned pair of first and 
second sequence sub-word units respectively; P(d ± |p r ) is 
the comparison score output by said first comparing means 
and is representative of the probability of confusing set 
5 sub-word unit p r as first sequence sub-word unit d 1 i ; 

P(d 2 j|p r ) is the comparison score output by said second 
comparing means and is representative of the probability 
y of confusing set sub-word unit p r as second sequence sub- 

word unit d 2 3 ; and P(p r ) is a weight which represents the 

4= 

y 10 probability of set sub-word unit p r occurring in a 

y sequence of sub-word units. 



Hhj- 37. A method according to claim 36, wherein said third 

«j comparing step identifies the set sub-word unit p r which 

5 

H 15 gives the maximum combined comparison score and wherein 

said determining step determines said sub-word unit 
representative of the sub-word units in the aligned pair 
as being the sub-word unit which provides the maximum 
combined comparison score . 

20 

38. A method according to claim 3 4 , wherein said 
comparison scores represent log probabilities and wherein 
said combining step multiplies said probabilities by 
adding the respective comparison scores. 
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39. A method according to claim 31, wherein each of the 
sub-word units in said first and second sequences of sub- 
word units belong to said set of predetermined sub-word 
units and wherein said first and second comparing steps 
provide said comparison scores using predetermined data 
which relate the sub-word units in said set to each 
other . 

40. A method according to claim 39, wherein said 
predetermined data comprises, for each sub-word unit in 
the set of sub-word units, a probability for confusing 
that sub-word unit with each of the other sub-word units 
in the set of sub-word units. 

41. A method according to claim 29, wherein said 
aligning and comparing step uses a dynamic programming 
technique to align said first and second sequences of 
sub-word units . 

42. A method according to claim 41, wherein said dynamic 
programming technique determines an optimum alignment 
between said first and second sequences of sub-word 
units . 
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43. A method according to claim 29 , wherein each of said 
sub-word units represents a phoneme. 

44. A method according to claim 29, further comprising 
receiving a third sequence of sub-word unit 
representative of a third one of the words output by said 
word recognition unit and wherein said aligning and 
comparing step simultaneously aligns and compares the 
sub-word units of the first, second and third sequences 
of sub-word units. 

45. A method according to claim 29, further comprising 
receiving a third sequence of sub-word units 
representative of a third one of said words output by 
said recognition unit and wherein said aligning and 
comparing step aligns and compares two sequences of sub- 
word units at a time. 

46. A method according to claim 29, further comprising 
the step of using a word to sub-word unit dictionary to 
convert the words output by said word recognition unit 
into said sequences of sub-word units. 
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47. A method according to claim 29, further comprising 
the step of annotating a data file using the sub-word 
units output by said determining step. 



5 4 8. A method according to claim 47, wherein said 

annotating step annotates said data file using said sub- 
word units and said words output by said word recognition 
unit . 

10 49. A method according to claim 48 , wherein said 

sequence of sub-word units and said words are combined to 
form annotation data for the data file. 

50. A method according to claim 47, wherein said data 
15 file is one of: an audio data file, a video data file, an 

image data file, or a text data file. 



51. A method according to claim 29, wherein said word 
recognition unit comprises a speech recognition system. 

52. A method apparatus according to claim 29, wherein 
said word recognition unit comprises a handwriting 
recognition system. 
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53. A method of determining a sequence of sub-word units 
representative of at least two words, the method 
comprising the steps of: 

receiving a first sequence of sub-word units 
representative of a first word; 

receiving a second sequence of sub-word units 
representative of a second word; 

aligning sub-word units of the first sequence with 
sub-word units of the second sequence to form a number of 
aligned pairs of sub-word units; and 

determining a sequence of sub-word units 
representative of the first and second sequences of sub- 
word units by determining, for each aligned pair of sub- 
word units, a sub-word unit that is confusingly similar 
to the first and second sub-word units of the aligned 
pair. 

54, A method of determining a sequence of sub-word units 
representative of at least two words output by a word 
recognition unit in response to a common input word to be 
recognised, the method comprising the steps of: 

receiving the words output by the word recognition 

unit ; 

generating a sequence of sub-word units 
representative of each of the received words; 
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aligning and comparing the sub-word units from each 
generated sequence of sub-word units to identify a number 
of aligned groups of sub-word units ; and 

determining a sequence of sub-word units 
representative of the received words in dependence upon 
the aligned groups of sub-word units determined in said 
aligning and comparing step. 

55. A method of determining a sequence of sub-word units 
representative of at least two words output by a word 
recognition unit in response to a common input word to be 
recognised, the method comprising the steps of: 

receiving the words output by the word recognition 

unit ; 

generating a sequence of sub-word units 
representative of each of the received words; 

aligning and comparing the sub-word units from each 
generated sequence of sub-word units to identify a number 
of aligned groups of sub-word units; and 

determining a sequence of sub-word units 
representative of the first and second sequences of sub- 
word units by determining, for each of the sub-word units 
in an aligned group, a sub-word unit that is confusingly 
similar to the sub-word units of the group. 
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56. A method of determining a sequence of sub-word units 
representative of at least two words output by a word 
recognition unit in response to a common input word to be 
recognised, the method comprising: 

receiving a first sequence of sub-word units 
representative of a first one of said at least two words; 

receiving a second sequence of sub-word units 
representative of a second one of said at least two 
words; 

aligning and comparing sub-word units of the first 
sequence with sub-word units of the second sequence to 
form a number of aligned pairs of sub-word units; 

a first comparing step of comparing, for each 
aligned pair, the first sequence sub-word unit in the 
aligned pair with each of a plurality of sub-word units 
taken from a set of predetermined sub-word units, to 
provide a corresponding plurality of comparison scores 
representative of the similarities between the first 
sequence sub-word unit and the respective sub-word units 
of the set; 

a second comparing step of comparing, for each 
aligned pair, the second sequence sub-word unit in the 
aligned pair with each of said plurality of sub-word 
units from the set, to provide a further corresponding 
plurality of comparison scores representative of the 
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similarities between said second sequence sub-word unit 
and the respective sub-word units of the set; 

combining the comparison scores obtained when 
comparing the first and second sequence sub-word units in 
the aligned pair with the same sub-word unit from the 
set, to generate a plurality of combined comparison 
scores ; 

a third comparing step of comparing, for each 
aligned pair, the combined comparison scores generated in 
said combining step for the aligned pair; and 

determining, for each aligned pair, a sub-word unit 
representative of the sub-word units in the aligned pair 
in dependence upon a comparison result output from said 
third comparing step for the aligned pair. 

57. A computer readable medium storing computer 
executable process steps to perform a method of 
determining a sequence of sub-word units representative 
of at least two words output by a word recognition unit 
in response to a common input word to be recognised, the 
process steps comprising the steps of: 

receiving a first sequence of sub-word units 
representative of a first one of said at least two words; 
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receiving a second sequence of sub-word units 
representative of a second one of said at least two 
words; 

aligning and comparing sub-word units of the first 
sequence with sub-word units of the second sequence to 
form a number of aligned pairs of sub-word units; and 

determining a sequence of sub-word units 
representative of the received sequences of sub-word 
units in dependence upon the aligned pairs of sub-word 
units determined in said aligning and comparing step, 

58. A computer executable program for controlling a 
processor to perform a method of determining a sequence 
of sub-word units representative of at least two words 
output by a word recognition unit in response to a common 
input word to be recognised, the program comprising: 

code for receiving a first sequence of sub-word 
units representative of a first one of said at least two 
words; 

code for receiving a second sequence of sub-word 
units representative of a second one of said at least two 
words ; 

code for aligning and comparing sub-word units of 
the first sequence with sub-word units of the second 
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sequence to form a number of aligned pairs of sub-word 
units; and 

code for determining a sequence of sub-word units 
representative of the received sequences of sub-word 
units in dependence upon the aligned pairs of sub-word 
units determined in said aligning and comparing step. 



